Towards real-time multilingual multimodal speech-to-speech translation

Speech-to-speech translation technology enables natural oral communication between different language speaking people. Many research projects have addressed speech-to-speech translation (S2ST) technology, such as ATR [1], VERBMOBIL [2], C-STAR [3], NESPOLE! [4], BABYLON [5], GALE [6], and EU-bridge [7]. The speechto-speech translation system is normally composed of automatic speech recognition (ASR), machine translation (MT), and speech synthesis (TTS). All of the modules are corpus-based and statistical model-based systems. In this talk, new challenges toward a real-time multimodal speechto-speech translation will be introduced.